Quantifying and reducing researcher subjectivity in the generation of climate indices from documentary sources

Abstract. The generation of index-based series meteorological phenomena, derived from narrative descriptions weather and climate in historical documentary sources, is a common method to reconstruct past climatic variability effectively extend the instrumental record. This study first explicitly examine degree inter-rater producing such series, potential source bias analyses. Two teams raters were asked produce five-category annual rainfall index for same dataset, consisting transcribed 11 “rain years” nineteenth-century Lesotho, originally collected by Nash Grab (2010). One group (n = 71) was comprised students studying postgraduate qualifications climatology or related discipline; second 6) consisted professional meteorologists climatologists working southern Africa. Inter-rater reliability high both groups at r 0.99 student 0.94 raters, although ratings provided disproportionately averaged central value (0: normal/seasonal rains) where high. Back calculation intraclass correlation using Spearman–Brown prediction formula showed that target 0.9 (considered “excellent” other published studies) could be obtained with as few eight four raters. number reduced two when examining subset 4) who had previously papers on We therefore conclude between researchers should considered minimal reconstructions are generated trained more.